Stream Processing Algorithms that Model Behavior Change

This paper presents algorithms that fuse information in multiple event streams to update models that represent system behavior. System behaviors vary over time; for example, an information network varies from heavily loaded to lightly loaded conditions; patterns of incidence of disease change at the onset of pandemics; file access patterns change from proper usage to improper use that may signify insider threat. The models that represent behavior must be updated frequently to adapt to changes rapidly; in the limit, models must be updated continuously with each new event. Algorithms that adapt to change in behavior must depend on the appropriate length of history: Algorithms that give too much weight to the distant past will not adapt to changes in behavior rapidly; algorithms that don’t consider enough past information may conclude incorrectly, from noisy data, that behavior has changed while the actual behavior remains unchanged. Efficient algorithms are incremental – the computational time required to incorporate each new event should be small and ideally independent of the length of the